Light field imaging relates to images created using light field photography, wherein the light field data, i.e. angular information, position, and radiance of light rays from the photographed object or scene are accessible from the resulting light field image.
A standard 2D camera captures the light rays in two dimensions at plane f. The radiance and the position are recorded as pixel values and pixel position (coordinates x and y). However, the angles of the light rays are not recorded.
However, a light field camera records angular information as well as radiance and position. Having data describing light rays as vectors the rendering of the image may be processed in a way that is not possible with a standard 2D camera.
In essence a light field image comprises at least two subset images, wherein each subset image corresponds to a different viewpoint or direction of the imaged object or scene. Each subset image is captured by a separate lens. Hence, angular information is recorded in relations or differences among the subset images.
Various solutions to the problem of capturing lightfield data have been presented. A 3D camera having two lenses can be considered a simple lightfield camera as some angular data is preserved. Arrays of multiple cameras have been used. The plenoptic camera uses a single main lens and an array of micro lenses located in proximity to the image sensor. The plenoptic cameras can be divided into two categories; the focused and the unfocused type.
FIG. 1 schematically illustrates a known focused plenoptic camera, which utilizes uses an array of microlenses, Lm, to capture the light field image from an object r, wherein the object r if captured at the focal plane f would result in a sharp image (as for a standard 2D camera). The image sensor (not shown) is placed in the focal plane f2 of the microlens array Lm. The light paths for two micro lenses are shown. Both microlenses picture the same point of the object by using just a portion of the total cone of light representing that particular point. A typical focused plenoptic camera uses about 20-50 lenses to capture the light bundles from a single point. A typical design has several thousand microlenses. The depth of field of the individual subset images formed by the microlenses is very large as the focal length of the microlenses is very short compared with the main lens L.
An object that is moved closer or farther away from the camera will be imaged at a different focal plane f as can be seen in FIGS. 2a and 2b. Hence, the different angles of the light rays translate to a difference in position in the microlens images.
FIG. 3a illustrates a portion of a light field image, captured by a plenoptic camera, of the two objects “C” and “F” with “C” positioned closer to the camera than “F”. Each hexagonal represents a subset image from a single microlens.
FIG. 3b shows two adjacent microlens subset images from the light field image of FIG. 3a. In FIG. 3c the two subset images has been superimposed in such a way that the letters C coincide. Note that the letter F is shifted sideways. This represents an image rendering with focus set at the distance of the letter C. In FIG. 3d the distance between the two images has been shifted to make the letter F sharp while the letter C is blurred. It can be seen from this example that a larger shift between the images (d) represents focus at closer distance from the camera. The out of focus portions in this example shows up as double strike letters. Normally when an image is rendered all microlens subset images are superimposed making out of focus objects appear smoothly blurred.
Another approach to rendering is to maintain the position of the individual microlens images but enlarging them until details in the desired focal plane coincide.
With current light field imaging techniques it is possible to focus, refocus, and change perspective. However an improved method, graphical user interface, and computer program product for processing of light field images allowing for new functionalities would be advantageous.